Skip to content

fix: route compress_prompt to llmlingua2 path when use_slingua=True (#235)#242

Open
ousamabenyounes wants to merge 1 commit intomicrosoft:mainfrom
ousamabenyounes:fix/issue-235
Open

fix: route compress_prompt to llmlingua2 path when use_slingua=True (#235)#242
ousamabenyounes wants to merge 1 commit intomicrosoft:mainfrom
ousamabenyounes:fix/issue-235

Conversation

@ousamabenyounes
Copy link
Copy Markdown

What does this PR do?

Fixes #235

When PromptCompressor is created with use_slingua=True, calling compress_prompt() raised:

TypeError: XLMRobertaForTokenClassification.forward() got an unexpected keyword argument 'past_key_values'

Root cause: compress_prompt() only routed to compress_prompt_llmlingua2() when self.use_llmlingua2 was True. __init__ already had the correct if use_llmlingua2 or use_slingua: guard (line 92) to call init_llmlingua2, but the dispatch in compress_prompt was missing the or self.use_slingua half.

Fix: one-line change at line 536 of llmlingua/prompt_compressor.py:

# Before
if self.use_llmlingua2:
# After
if self.use_llmlingua2 or self.use_slingua:

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Was this discussed/approved via a Github issue? Please add a link to it if that's the case.
    Issue: [Bug]: When calling slingua through PromptCompressor, use_llmlingua2 should also be set to true. #235
  • Did you make sure to update the documentation with your changes? (not applicable — no new API)
  • Did you write any new necessary tests?
    Added tests/test_slingua_routing.py with 3 unit tests covering the fixed routing, existing llmlingua2 routing, and the unchanged neither-flag path. Tests use sys.modules stubs so they run without ML dependencies.

Who can review?

@iofu728 @QianhuiWu @XufangLuo @mydmdm

Generated by Claude Code
Vibe coded by ousamabenyounes

…icrosoft#235)

When use_slingua=True, __init__ already called init_llmlingua2 correctly,
but compress_prompt only checked self.use_llmlingua2 and fell through to
the LLMLingua-1 causal-LM path, which passes past_key_values to
XLMRobertaForTokenClassification and crashes.

Generated by Claude Code
Vibe coded by ousamabenyounes

Co-Authored-By: Claude <noreply@anthropic.com>
@ousamabenyounes
Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: When calling slingua through PromptCompressor, use_llmlingua2 should also be set to true.

1 participant